1. Field of the Invention
The present invention relates to an apparatus for performing moving picture encoding decoding processing, especially an apparatus which uses a motion vector for the encoding process.
2. Description of the Related Art
Recently, the technical development of the digital telecommunication system has progressed remarkably, and various technologies such as multimedia communication and the personal mobile communication are discussed and researched actively. Various types of data are used and transmitted in such multimedia communication including voice data, moving picture data, and still image data. Therefore, high speed communication technology for picture data will be a key and important technology in the forthcoming multimedia communication system. Because the data amount of image data is huge, the data encoding decoding technology is an important technology. As for the encoding decoding algorithm, there are MPEG1, MPEG2, H.261 and H.263 in ITU-T etc. Standardization of those algorithms is discussed and determined according to the usage and purpose.
In these encoding decoding algorithms, an intraframe encoding process including orthogonal transformations and an interframe encoding process including motion compensation are performed. Through those encoding decoding processes, a process that relates to the motion vector detection occupies the major part of the whole processing in the algorithm. The data encoding decoding using the motion vectors is indispensable processing for the improvement of the picture quality. Interframe encoding first calculates a data difference between former image data and the current input image and performs encoding using the calculated interframe difference (as shown in FIG. 16, data difference 1 is calculated based on a former image data al and an input image data b). If a data difference becomes small, encoding efficiency becomes large. However, when image data with much motion is compressed, the interframe difference becomes large. In this case, use of interframe difference processing will not contribute to improve the encoding efficiency very much. Then, the encoding efficiency can be improved by performing motion compensation preliminarily, where former image data is moved based on the forecasting of the movement of former image data, and then performing interframe data difference processing (as shown in FIG. 16, difference 2 is calculated based on an amended data a2 from former image with motion compensation and an input current image data b). This movement direction with forecasting image data motion is defined as a motion vector.
Hereafter, the outline of the motion vector detection is shown below. The compressed algorithm performs encoding process by a macroblock unit which composes 16 pixelsxc3x9716 pixels matrix data, and the motion vector is detected by the macroblock unit. The macroblock for calculating motion vector is moved around by a pixel length step in the search area (FIG. 17(a)). Next, the correlation of the moved macroblock and the corresponding macroblock of the input image is calculated. The correlation is calculated as the absolute value harmony of the respective difference of the corresponding pixel data.
The absolute value harmony of the difference of the macroblock is calculated respectively after moving in all the search areas, and the maximum correlation is detected among the calculated correlation, and the amount of the movement of the detected moved macroblock is assumed to be a motion vector.
However, if for example macroblock 200 located on the boundary of the frame comes to the target macroblock for detecting motion vector as shown in FIG. 17 (b), the search area 210 for the macroblock 200 covers the area beyond the frame. In a conventional encoding process, such a motion vector was supposed not to be handled. When an image picture moves in the direction where actual image data go outside of the frame, motion compensation cannot be used for the interframe encoding. So the encoding efficiency has dropped remarkably. Moreover, the picture quality might worsen as a result due to the noise such as block distortions. Then, unrestricted motion vector is added as an option in H.263 etc. in order to raise accuracy. The unrestricted motion vector is used for calculating the motion vector when actual image data go outside of the frame, assuming that whole data of the search area 210 exist by providing pixel data to the vacant area of the search area 210 outside of the frame. When correct image data for outside area cannot be obtained from the frame data, the data of frame boundary is assumed to be the nearest data. As for the implementation method for achieving the above mentioned unrestricted motion vector calculation processing, the following implementation methods normally can be considered. The first implementation method is the software processing without dedicated hardware such as LSI. According to the flow chart as shown in FIG. 4, with the pixel data moving, the address after moving is determined to be within the frame or outside the frame by adding or subtracting moving distance to the address before moving. If the address after moving is outside the frame, the pixel data difference is processed based on the nearest boundary pixel data of the frame and the input current pixel.
The second implementation method is the hardware by expanding the set frame memory area in order to cover all search area composed of the frame data area and the outer surrounding area, and all data of the search area which are the frame data and the outer surrounding area data are transmitted precedingly, and correlation processing is performed based on the data on the expanded frame memory and the input data.
However, according to the above mentioned first implementation by software means, when motion vector calculation process for the macroblock, the conditional branch processes are included in all calculation every after moving around in the search area, the processing becomes complex. In addition, if a processor uses the pipeline processing, the so called xe2x80x9cpipeline penaltyxe2x80x9d is caused when the conditional branch process occurs and the processing efficiency deteriorates greatly.
Moreover, according to the above mentioned second implementation by hardware means of expanding memory area, the necessary memory capacity consumption becomes large.
Therefore, with the foregoing in mind, it is an object of the present invention to provide a moving picture data encoding decoding processing apparatus which can detect the unrestricted motion vector with a dedicated memory having a small capacity without increasing the software process.
In order to achieve the objects, a moving picture encoding decoding processing apparatus includes a frame memory, a prediction memory for storing the image data of a search area for a motion vector detection, an image data transmission means for transmitting image data of a first macroblock and surrounding image data existing in the frame memory from the frame memory to the prediction memory and writing the transmitted data to the corresponding area wherein the first macroblock is the macroblock whose search area extends over the frame image area, an image data extrapolation means for forming an image data to the extrapolation area of the search area where an image data had not been written onto the prediction memory by extrapolation based on the pixel data located on the boundary with the search area, and a motion vector detecting part for detecting the unrestricted motion vector based on the image data of the search area formed on the prediction memory.
This embodiment makes it possible to form the image data for the search area onto the prediction memory rapidly and efficiently and to detect unrestricted motion vector. The image data for the search area in the prediction memory is formed by transmitting and writing the first macroblock currently processed as a target for encoding and surrounding image data of the first macroblock from frame memory to the prediction memory and writing the same data with the image data positioned on the boundary data of the first macroblock to the rest of extrapolation area. The memory consumption capacity of the prediction memory can be reduced to the capacity for the first macroblock and the surrounding image data of the first macroblock.
Next, the moving picture encoding decoding processing apparatus of the present invention preferably includes a macroblock recognition part for recognizing whether an input macroblock is a macroblock whose full motion vector search area is within the frame image data area or a macroblock whose motion vector search area has at least a part extending over the frame image data area. The motion vector detection part preferably detects the motion vector based on the image data formed on the frame memory if an inputted macroblock is the macroblock whose full motion vector search area is within the frame image data area, and detects the motion vector based on the image data formed on the prediction memory if an inputted macroblock is a macroblock whose motion vector search area has at least a part extending over the frame image data area.
This embodiment makes it possible to perform the motion vector detection by using the prediction memory only when the currently processed macroblock is a macroblock whose search area is extending over the frame image area. The processing amount for the motion vector detection can be reduced and the process can be performed efficiently.
Next, the macroblock recognition part preferably recognizes an inputted macroblock as a macroblock whose motion vector search area is extending over the frame image data area if the position of the inputted macroblock is in a surrounding edge area of the frame image. The macroblock recognition part preferably recognizes an inputted macroblock as a macroblock whose full motion vector search area is within the frame image data area if the position of the inputted macroblock is in an internal area excluding the surrounding edge area of the frame image.
This embodiment makes it possible to detect whether the prediction memory is used for detecting the motion vector for the inputted macroblock. It can simply detect by the macroblock number whether the macroblock is located on the surrounding edge of the frame image or is located on the internal side of the frame image without a complex conditional branch sequence. Here, the macroblock positioned on the surrounding edge means the macroblock located near the edge of the frame image and having the search area extending over the frame image area. The inputted macroblock uniquely is detected as the surrounding edge macroblock based on its location and set length of the search area.
Next, the prediction memory is preferably composed of (N+2M)xc3x97(N+2M) capacity of memory when the macroblock is composed of Nxc3x97N pixels and the search area has M data pixel length to upper side, lower side, right-hand side, left-hand side. This embodiment makes it possible to provide the prediction memory with an appropriate capacity based on the set search area length. By this way, the search area can be set appropriately and memory consumption capacity can be reduced by providing a necessary amount of memory for the prediction memory.
Next, the image data extrapolation preferably includes a first extrapolation means for writing the pixel data located on boundary with the search area to the same row address of the extrapolation area in which image data are positioned in the nearest pixel to the extrapolation area when inputted macroblock image data are mapped on the prediction memory if there are image data having the same row address as the extrapolation area in the inputted first macroblock data. The image data extrapolation also includes a second extrapolation means for writing the pixel data located on the boundary with the search area to the same column address of the extrapolation area in which image data are positioned in the nearest pixel to the extrapolation area when inputted macroblock image data are mapped on the prediction memory if there are image data having the same column address as the extrapolation area in the inputted first macroblock data. Moreover the image data extrapolation preferably includes a third extrapolation means for writing the corner pixel data located on the boundary with the search area to the faced extrapolation area which image data is positioned in the nearest corner pixel to the extrapolation area when inputted macroblock image data are mapped on the prediction memory if there are no image data having the same row address or the same column address as the extrapolation area in the inputted first macroblock data.
This embodiment makes it possible to perform extrapolation by specifying the image data of the frame image which is located the nearest to the extrapolation area uniquely as image data for extrapolation and to form the image data for motion vector detection onto the prediction memory.
Next, the macroblock recognition part preferably outputs an extrapolation pattern signal indicating the position of the macroblock in the surrounding edge if the inputted macroblock is in the surrounding edge. The image data extrapolation means preferably performs the extrapolation based on the extrapolation pattern.
This embodiment makes it possible to specify the extrapolation pattern by the extrapolation pattern signal. The extrapolation pattern is uniquely decided based on the position of the inputted macroblock in the frame image. The prediction memory can perform the extrapolation simply and rapidly and form the image data of search area onto it.
Next, the image data extrapolation means preferably includes a row decoder generating one or plural set of row addresses for the prediction memory by decoding based on the address signal indicating the accessing address of the prediction memory, the extrapolation pattern signal and the read/write signal for the prediction memory. Moreover, the image data extrapolation means preferably includes a column decoder generating one or plural set of column addresses for the prediction memory by decoding based on the address signal indicating the accessing address of the prediction memory, the extrapolation pattern signal and the read/write signal for the prediction memory.
This embodiment makes it possible to specify the address of the prediction memory for the first macroblock to be written. Moreover, the extrapolation means of the prediction memory can specify the row address and column address of the extrapolation area for image data extrapolation based on the extrapolation pattern signal. The image data for the search area can be formed onto the prediction memory rapidly.
These and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.